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ABSTRACT 



The relationship to readability of ten variables 
characterizing structural properties cf mathematical prose was 
investigated in elementary algebra textbooks. Readability was 
measured by algetra students responses to two forms of cloze tests. 
Linear and currilinear correlations were calculated between each 
structural variatle and the cloze test. The results indicated that 
explanatory material, sentence length, percentage of mathematical 
symbols and percentage of difficult words had significant linear 
correlations with readability. In illustrative material, percentage 
of mathematical symbols and percentage of mathematics vocabulary had 
significant linear correlations with readability. Departures from 
linearity were significant for sentence length and percentage of 
reader-directed sentences in explanatory material, and percentage of 
difficult words in illustrative material. [Not available in hardcopy 
due to marginal legibility of original document. } (Author/CT) 
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In the past, Investigations of the readability of mathematical 
English have been extremely limited. Most studies have attempted 
to apply to mathematics readability formulas that were developed 
for use with ordinary English. Kane (1970) has outlined the 
difficulties Inherent In this approach. Vocabulary differences 
between mathematical English (ME) and ordinary English (OE) make 
Inapplicable formulas that use a count of words not on a list of 
familiar words. Words that are familiar In OE may have special 
meanings In ME and, hence, may not be familiar. Also words 
that are familiar In ME may not appear on the list. A more 
fundamental reason that the foi'mulas are not applicable Is that 
they apply only to prose portions of the text. The readability 
of symbolic material Is completely neglected In these formulas. 

The purpose of this study was to investigate structural 
variables that might predict the readability of ME as found In 
flrct-year algebra textbooks. A large portion of school mathe- 
matics textbook material oan be classified either as explanatory , 
in vhloh a concept Is defined and explained, or as illustrative , 
in whioh speolflo problems or exercises that illustrate a concept 
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are worked out for the reader. These two types of ME were 
investigated separately to determine If their difficulty was 
predicted differently by the structural variables. The structural 
variables were also Investigated to see If their relationship 
with readability was linear. If the relationship is linear 
for a particular variable, then Its predictive value is applicable 
over a range of readability levels. 

The criterion of difficulty is of critical Importance in 
any readability study. The most common criterion Is the score 
on a test, usually multiple-choice, over the content of the 
passage being analyzed. The most obvious drawback of this criterion 
is that difficult questions can be constructed, for easy passages 
and vice versa. Many readability formulas use this criterion 
because nationally normed tests, such as the McCall-Crabbs, 
are available for OE passages, The lack of such passages and 
tests for ME is one reason that readability formulas for KE have 
not been developed. 

Reading difficulty can also be obtained by having Judges rate 
the passages. This method was used for OE by Elley (1969) with 
promising results. He founu that Judges' ratings correlated 
highly with structural variables and with readability formula 
scores. Judges' ratings are muoh simpler to obtain than other 
measures that have been used as orlteria. Rankings for a large 
number of passages, however, are difficult to obtain. Elley 
obtained rankings of only 16 passages and he was unable to build 
an Interval scale of difficulty by paired comparisons. 
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In a study of mathematical word problems, Kilpatrick (i960) 
used the subjects' ability to write the appropriate equation 
for a problem as the readability criterion. This method avoids 
the disadvantages of comprehension tests, but It Is difficult 
to Interpret when a partially correct equation is produced. 

The method Is, of course, not applicable to other types of ME 
passages, 

A readability criterion that has gained wide acceptance is 
the cloze procedure, developed by Taylor (1953), A cloze test 
Is constructed by deleting words from a passage and inserting 
blanks. The reader Is Instructed to replace the words, using 
clues from the remaining context. The number of words replaced 
incorrectly, averaged across readers, is taken as a measure of 
the passage's difficulty. 

The cloze procedure is attractive because of Its objectivity, 
and it correlates highly with other measures of readability 
(Bormuth, 19665 Hafner, 1963; Rankin, 1964), However, cloze 
tests are laborious to construct. Furthermore, although no one 
ha8 investigated the question, mutilating a passage by inserting 
blanks may, particularly in ME, Increase reading difficulty because 
of the unusual appearance of the text. 

Despite these limitations, the cloze test can be applied to 
ME, including symbolic and graphical material, if special agree- 
ments are made, Each graphic sign In a passage must be classified 
as a single Item for deletion. In the following discussion, the 
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term "token" refers to an occurrence of a sign (word, algebraic 
symbol) In context and the term "type" refers to the listing 
of a sign as In a dictionary. The number of tokens In a passage 
Is simply the number of signs that occur; the number of types 
Is the number of different signs that occur. 

Signs In OE are easily classified Into word tokens: a token 
is separated by spaces from other tokens. In general, a word 
token Is eligible for deletion If there is a space before and 
after It. Although there Is no simple definition of a math token, 
Hater (1969) showed how to Identify Individual tokens In an 
ME passage. Math tokens usually do not have a phoneme-grapheme 
relation to spoken words, nor do they always have a one-to-one 
correspondence with words. Generally, a math token Is the 
smallest unit that Independently conveys the Intended meaning of 
the part of the passage In which It occurs. In an ME passage, 
a math token Is any sign that is not a word token, punotuatlon, 
or drawing. Each word token and math token Is eligible for dele- 
tion, lnoludlng tokens that appear in tables, charts, on graphs, 
and as labels for drawings. 

The validity of the doze test as a readability criterion for 
ME was investigated by Hater ( 1969 )* who used five forms of the 
cloze test for each of five passages. Average doze soore was 
highly correlated (r » ,83) with reading difficulty as measured 
by multiple-choice tests. Single forms of the doze test were 
not highly correlated with reading difficulty. However, all but 
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one form ranked the passages In the same order of difficulty. 

Hater recommended use of the cloze procedure as a measure of the 
readability of ME. 

Definitions of Structural Variables 

The unique nature of mathematical English cs a combination 
of a natural language (English) and other languages, such as 
sentential calculus, algebraic notation, and the like (Kane 1970), 
makes It necessary to consider additional structural variables 
beyond those commonly used In readability formulas for ordinary 
English, 

Average sentence length 

Since ME Is written In English, it Is reasonable to expect 
that some structural variables of English are Important determiners 
of difficulty. In many readability formulas, a measure of 
sentence length accounts for a large portion of che reading 
difficulty of a passage. The definition of sentence length In 
this study was the average number of words per sentence in the 
passage. The measure wa3 obtained by counting the number of word 
and math tokens and dividing the total by the number of sentences 
in the passage. 

Percentage of personal words 

A personal Interest or concreteness measure has been used 
in some formulas, Plesoh (1950), for example, used the percentage 
of "definite words" which had a higher correlation with comprehen- 
sion than did sentence length. The percentage of personal words 
— like Plesoh's variable -- was Intended to provide a measure 
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of the abstractness of ME. Personal words Include names of 
people; common nouns with a natural gender, such as "father"; 
personal pronouns; and possessive pronouns. The measure was 
obtained by counting the personal word tokens and dividing the 
total by the number of word tokens. 

Percentage of difficult word s 

There are two general procedures for determining word difficulty. 
One Is to determine the average length In letters or syllables 
of the words of a passage. The other Is to determine the percentage 
of words In the passage that are not on a previously prepared 
list of familiar words. The latter procedure was used In this 
study since word length seemed less able to differentiate between 
ordinary English words and mathematics vocabulary words. The 
original Thorndike 1000 word list was used (Thorndike & Lorge, 

1944), The measure was obtained by counting each word token 
that was not on the list and dividing the total by the number of 
word tokens in. the passage. 

Percentage of mathematics vocabulary words 

As pointed out earlier, a major drawback in applying existing 
readability formulas to ME Is that they do not account for Its 
specialized vooabulary, A list of familiar mathemat? lal vocab- 
ulary words does not exist although work Is In progress to develop 
one. Without such a list, the alternative Is to count all of 
the mathematical vocabulary words In the passage. In a study of 
mathematical vocabulary, Byrne (19?0) gathered words from mathe- 
matics texts for grades 4 through 12 and compiled a list of 1165 
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words and phrases. The percentage of mathematical vocabulary 
words In this study was obtained by counting each word token 
that appeared on the Byrne list and dividing the total by the 
number of word tokens. 

Percentage of mathematics symbols 

Heavy use of symbolism Is an obvious characteristic of ME. 
Symbolism In the form of well-known formulas and In the detailed 
solution of equations Is highly redundant and should aid reading. 

On the other hand, symbolism Is extremely concise and may decrease 
readability because of Its technical and abstract nature. In 
either case, this variable seemed a reasonable one to si;udy. 

The measure was obtained by dividing the number of math tokens 
by the total number of tokens in the passage. 

Number of Implicit numbers, operations, and relations 

In a passage about a mathematical concept, It may be Important 
whether the processes Involved are stated In words or In symbols. 

For example, the statement "The sum of 6 and 2 Is 8" can be 
expressed symbolically as "6 + 2 = 8", where the , '+ ,, and the "=" 
are Implicit symbols and must be Inferred from the statement. In 
a reading study, Davis (196?) found that the ability to draw 
Inferences from content accounted for a large portion of the 
variance of reading skills, ranking second only to recall of word 
meanings. The variable used In this study to refleot this charac- 
teristic of HE Is the number of implicit numbers, operations, 
and. relations. In Kilpatrick's study (I960), the variable contributed 

significantly to the prediction of the readability of word problems, 
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The measure Is obtained by noting occurrences of processes 
that are stated in words or implicitly given in the context. 

The appropriate translation to symbolic form is made, and each 
occurrence of a number, operation, or relation that does not 
appear explicitly in the passage Is counted. 

The last four variables were selected because of their 
ability to refleot the reading difficulty of abstract prose 
material. 

Percentage of questions 

In studies of written material, Frase (196?) and Rothkopf 
and Bisblcos (1967) found that the number and position of questions 
in a passage affects learning, Davis (1967) identified the 
ability to find answers to explicit questions as an Important 
reading skill. The percentage of questions was obtained by 
counting the number of questions in the passage and dividing by 
the number of sentences. 

Percentage of reader-directed sentences 

Abstract written material may be less difficult to read if 
the author attempts a direct contact or communication with the 
reader. Abstract prose frequently contains many passive verbs 
and other grammatical forms that can cause difficulty in interpre- 
tation. Some investigators of readability have attempted to 
Identify grammatical olasses that refleot stylistic complexity. 

If written material is directed to the reader, the style will 
be "simple* in the sense that relative clauses, passive sentences, 
other less "direot" sentence constructions are not used. 
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The measure used In this study was obtained by oountlng the 
number of sentences containing a form of the pronoun "you," 
the number of Imperative sentences, and the number of non- 
rhetorlcal questions, and dividing by the total number of 
sentences In the passage. 

Number of gr arh lc clues 

In ME passages, It Is fairly common to find graphs, tables, 
and other pictorial material that are not classified as word or 
math tokens. It Is clear that these are Intended as reading 
aids and should, therefore, be Included as structural features 
of MS that may affect readability. The measure cf the variable, 
number of graphic clues, was obtained by counting each occurrence 
of a graph, number line, table, diagram, or other- symbolic 
materiel that Is not a word token, math token, or punctuation. 
Percentage of connected sentences 

A passage In which the Ideas follow closely may be highly 
readable. On the other hand, If one sentence Is difficult, the 
following ones may also be difficult. Percentage of connected 
sentences Is a variable designed to measure the extent to which 
a sentence depends on the one preceding. The measure v/as obtained 
by counting the number of sentences that begin with "linkers'' 
such as "and," "but," "since" and so on, In addition to sentences 
that began with adverbial phrases or olauses. The number of these 
eentenoer was divided by the total number of sentences In the 
passage. 
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Methods and Procedures 

The passages for the study were selected from nine elementary 
algebra textbooks. Explanatory (E) passages were chosen that 
presented a single concept. The $0 E passages represented a 
wide range of topics and difficulty. For the analysis of 
structural variables, the first 250 tokens were used. On the 
tests constructed from the passages, the entire sentence containing 
the 250th token was Included. The 50 illustrative example (X) 
passages were selected from the same nine textbooks. The X 
passages presented the solution of equations, word problems, 
factorizations, and other algebraic problems. The examples were 
chosen to represent a variety of types of exercises. Since 
solutions are limited In length, each X passage contained 1?5 
tokens . 

For each passage, two forms of the doze test were constructed! 
Form A by deleting the first and every fifth following token, 
and Form B by deleting the third and every fifth following token. 
Each test over the E passages contained 50 Items, and each test 
ov the X passages contained 35 Items. Hater's results (1969) 
suggested that two forms would be adequate. 

The tests were given to 860 first-year algebra students In 
four high schools in New York State. The students were of average 
ability. Some were tenth and eleventh graders, but most were 
in the ninth grade. One of the textbooks used to seleot passages 
was being used In three of the schools; the other eight textbooks 
were not being used. 
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Bach student received a packet containing directions, an 
E test and an X test. In half of the packets, the E test was 
first, and all possible combinations of test form pairings were 
evenly distributed. Each form of each test was given to students 
in at least two of the schools, and most were given to students 
in all four schools. The test period was 30 minutes. The 
regular classroom teacher administered the tests using directions 
provided by the Investigator, All testing was carried out during 
the week of November 16 to 20, 1970. 

Data end Analysts 

For four of the ten structural variables (percent of reader- 
directed sentences, number of graphic clues, percent connected 
sentences, and number of implicit symbols ), values wer.e 
determined by a second person a3 well as by the Investigator, 
and the average of the values was used. The mean values of each 
of the structural variables for the E and X passages are given 
In Table I. 

The values In Table I indicate some of the structural differences 
between E and X passages. The striking significant difference 
In MT was e/peoted since many of the X passages were almost 
completely symbolie. The shorter average SL of X passages was 
also expected since msny of the sentences of the X passages 
consisted of equations, and at this grade level the equations are 
quite short and simple. The difference in CS evidently reflected 
the contrast between E passages, whloh were continuous discourse, 
and X passages, which were composed of linguistically disconnected 
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Means and standard deviations of structural variables and 
difficulty, and t-values for test of differences between 
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steps In the solution of equations. The remaining significant 
difference was in D3. This difference probably reflects the 
combination of imperatives and direct questions in X passages. 

The readability criterion used in the study was obtained by 
averaging the means of the Form A and Form B cloze test scores 
for each passage. The reliability of the combined score was 
computed by an analysis of variance. For the E passages, the 
single form of reliability was 0,57« end the combined forms 
reliability was 0.73. For the X passages, the single form 
reliability was 0.63, and the combined forms reliability wa3 
0.77. The combined form reliabilities are high enough for 
comparisons of overall combinations of passages, but not for 
comparisons of single passages. Th*^ single form reliabilities 
are close to the correlation found by Hater ( 1969 ) between a 
single form of a cloze test and comprehension test scores (0.5*0. 
Evidently, combination of two forms Is almost a? reliable as 
the combination of five forms. 

The F-ratlos in Tables II and III Indicate that for 2 and 
X passages the two forms were not significantly different In 
difficulty and that the passages were significantly different 
in difficulty. 

Prcduot moment correlation coefficients and correlation 
ratios were used to investigate the relationship between 
readability and each of the structural variables. Eta values 
were computed for the regression of the structural variables 
on readability, using six Intervals of reading difficulty. 
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Table II 

Test by Form ANOVA for S Passages 



Source 


Sum of squares 


df 


Mean square 


F 


Between tests 


1846.7 


49 


37.69 


3.69* 


Between forms 


21.4 


1 


21.40 


2.09 


Error 


501.1 


49 


10.22 




Total 


2369.2 


99 






* 






* 


►0 

A 

• 

0 




Table III 










Test by Form ANOVA 


for X Passages 




Source 


Sura of squares 


df 


Mean square 


F 


Between tests 


1417.65 


49 


28.93 


4.37* 


Between forms 


0.34 


1 


0.34 


0.05 


Error 


327.61 


49 


6,62 




Total 


1745.26 


99 







*p< .01 



ERiC 



14 



15 



Correlation coefficients calculated using array means are 
usually higher than those using individual scores. This must 
be remembered in interpreting the values of r in Tables IV and 
V. For each variable, F-ratlos, Fi, F 2 . and F-j, were computed- 
to test the significance >f the correlation ratio, eta; the 
linear correlation coefficient, r; and the departure from 
linearity, respectively. 

The relationships between the structural variables and 
readability were different for the E and X passages. One 
variable, percent of math tokens had a significant linear cor- 
relation with readability in both types of passages. This 
variable alone would provide a fair prediction of the readability 
of tnese two types of elementary algebra passages. Average 
sentence length was significantly correlated with the readability 
of E passages, but thei'e was also a significant departure from 
linearity. Apparently, the shorter sentence length of X passages 
reduced the Importance of the variable for this type of material. 
The percentage of difficult words (DW) had a significant negative 
linear correlation with the reading difficulty of E passages. 
Although DW did not have significant linear correlation with 
readability in X passages, the correlation ratio and the depar- 
ture from linearity were significant. The nature of this 
relationship will be discussed later. Another contrast of the E 
and X passages was provided by the math vocabulary measure. In 
the X passages, the linear correlation of MV and readability 
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’roduct moment and eta correlations of X passage difficulty 
with each structural variable, and the F ratios obtained 
from tests of significance of correlation ratio, linear 
correlation, and departure from linearity* (N = 50) 
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was highly significant, but In the E passages the correlation 
only approached the .05 level of significance. The passages 
did not differ signlf lcantly In percentage of math vocabulary, 
so the differences In the correlations must have been due to the 
nature of the passages. The Investigator expected that math 
vocabulary would be an Important predictor; but he would have 
predicted the correlations with readability for the two types 
of passages to be the opposite of what they are. The continuous 
discourse In the E passages may permit the meaning of relatively 
more words to be Inferred from the context, thereby reducing the 
Impact of vocabulary. In contrast, many of the math vocabulary 
words In the X passages are Independent of other verbal material 
and consist of phrases, such as "commutative property of multipli- 
cation," used to Justify particular algebraic operations used 
In the solution of a problem. Another explanation may be that 
In E passages, the names of numbers, such as "two," and operations, 
such as “sum," are written as word tokens and are thus counted as 
math vocabulary Items, In X passages, these words are usually 
written In symbolic form, as "2" and "+." The result Is that 
although the percentage of math vocabulary words may be the same, 
the E passages may contain many easy math vocabulary words. Math 
vocabulary Items In the X passages are likely to be more difficult. 
Once data are available on the familiarity of words and phrases 
In the Byrne list, the math vocabulary variable may be adapted 
to yield a more sensitive measure. 
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/ N6ne of the variables representing facilitation of reading 
were significantly correlated with readability. The percentage 
of questions approached the .05 level In the X passages, A 
more refined measure that reflected the type and position of 
questions might yield a significant positive correlation. In 
the E passages, the percentage of reader-directed sentences was 
positively correlated with readability. Although r was not 
significant, the correlation ratio was, Indicating that the 
relationship may be curvilinear. Percentage of personal words 
had a small positive linear correlation with readability. In the 
X passages, the correlation approached the ,05 level. 

Additional Information on the relationships between structural 
variables and readability Is provided by plots of the means of 
the arrays of those variables exhibiting a significant departure 
from linearity. The number of arrays is too small to characterize 
the curves precisely, but some inferences can be made about the 
relationships. The plots are given in figures 1 to 4. 

Figure 1 suggests that In E passages readability Is relatively 
unaffected by sentence length until very long sentences are 
encountered. Attempts to decrease reading difficulty in ME 
passages by decreasing the sentence length may therefore have 
little effeot once extremely long sentences have been eliminated. 

The nature of the relationship In Figure 2 between RD and the 
difficulty of E passages Is apparently due to two passages that 
have high RD and high difficulty. In a larger sample, the 
relationship may be linear. 

In Figure 3 there apoears to be an unmistakable curvilinear 

0 

(| relationship between DW and the difficulty of X passages. However, 



Percent difficult words Average sentence length 
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Figures 1-4, Plots of the means of structural variables 
which have non-linear relationships with readabilty. 
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the number of passages In the very difficult and very easy 
intervals Is small. Also, the relationship Is linear for E 
passages. The ourvillnear relation in Figure 3, therefore, 
should be Interpreted cautiously. If the curvilinear relation- 
ship between DW and difficulty Is a true characteristic of X 
passages, the prediotlve value of the variable Is questionable 
since the same value can correspond to an easy or a difficult 
passage. 

The relationship between personal words and difficulty of 
X passages approaches significant curvllinearlty. Since the 
average value of PW Is only about In ME passages, slight 
variations may account for the nonlinear nature of the graph In 
Figure’ 4. It Is possible that higher values of PW could produce 
a more definite relationship, but personal words are not used 
extensively in ME passages. 

Summary and Conclusions 

The relationship to readability of ten variables characterizing 
structural properties of mathematical prose was Investigated for 
50 explanatory (E) passages and 50 Illustrative example (X) passages 
from elementary algebra tests. The E passages had greater average 
sentence length and more connected sentences. The X passages had 
a greater percentage of math tokens and reader- directed sentences. 

Readability was measured by algebra students' respoi.ses to two 
forms of doze tests over eaoh passage. The combined forms had 

O 
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reliability coefficients of 0,?3 and 0.7? for the E and X 
passages respectively. Linear and curvilinear correlations were 
calculated between each structural variable and the cloze test. 

In E passages, sentence length, percentage of math tokens, and 
percentage of difficult words had significant linear correlations 
with readability. Ir, X passages, percentage of math tokens and 
percentage of math vocabulary had significant linear correlations 
with readability. Departures from linearity were significant for 
sentence length and percentage of reader-directed sentences In 
E passages, and percentage of difficult words In X passages. 

The results of the study confirm the view that MS is structurally 
different from OE. The sentence length and difficult word 
vsi'isteliic cor.sofily uo6& lx* 02 llty f o I'm vs 1 ® s w 9 r 9 not 

similarly related to the readability of HE passages. Both 
variables showed evidence of curvilinear relationships with the 
readability of ME but there relationships may be due to the small 
number of passages In the extreme Intervals of difficulty. 

The structural variables Investigated In this study show promise 
of providing a significant combined prediction of the readability 
of elementary algebra ME passages. A multiple regression analysis 
will be performed using the ten structural variables as predictors 
and the cloze te?t as the readability criterion. T:ie results of 
suoh an analysis, done separately for the explanatory and 
Illustrative passages will provide specific information about 
hew these two types of algebra passages differ from each other 
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and from ordinary English, 

At the present time, Intuition, experience, and knowledge 
of readability factors of ordinary English provide the only 
approach to determining the readability of mathematical English. 
It is apparent that the two types of elementary algebra passages 
investigated in this study have distinct and somewhat different 
structural characteristics that are related to readability. 
Further analysis will provide specific Information about the 
relative weights of the structural variables In relation to 
readability. In the future, more refined measures of variables 
such as mathematics vocabulary nay yield Insight into the nature 
of mathematical English. When precise determinations of these 
variables are made, mathematical English can be written with an 
awareness of specific existing reading difficulties. With this 
knowledge, attempts to make mathematical English readable promise 
to be more effective. 
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